gamma-C plane and robustness in static reservoir for nonlinear regression estimation

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting galley proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. A c c e p t e d m a n u s c r i p t Abstract: Reservoir method is applied to the feed-forward learning machines for nonlinear regression estimation. Inspired by the existing experience from extreme learning machine (ELM), the new method inherits the basic idea from support vector echo-state machines, but eliminates the internal feedback matrix to adapt for the feed-forward usage. Based on the analysis of nonlinearity in reservoir and regularization in readout weights, the parameters of input scaling and penalty regularization are taken as the hyper-parameters to characterize a static reservoir (ELM); and then a proper reservoir is identified on the J-C plane based on a generalization error criterion. For outlier suppression, the regularized robust regression is applied in the reservoir feature space, and it leads to an efficient algorithm for large-scale problems, which can be solved by Cholesky decomposition. The proposed method is compared with the classical kernel method and ELM method on several benchmark nonlinear regression datasets, and the results indicate the method is comparable with the existing methods.